Software applications present on a computer system, and in particular those applications that access the Internet, are vulnerable to attack. For example, a web browser application may be attacked when that application is used to download and view a web page from a remote server. An attacker may include so-called “shellcode” inside the web page and which comprises machine-code instructions that are written into the browser memory space. A remote shellcode is useful if an attacker wants to “browse around” the target system, but more often an attacker will just want to install some form of malware on the target system. In such cases, download and execute shellcode is often used: this type of shellcode does not spawn a shell, but rather instructs the machine to download a certain executable file off the network, save it to disk and execute it. Nowadays, this approach is commonly used in drive-by download attacks, where a victim visits a malicious webpage that in turn attempts to run such a download and execute shellcode in order to install software on the victim's machine.
An attacker will commonly inject a shellcode into the target process before or at the same time as it exploits a vulnerability to gain control over the program counter. The program counter is adjusted to point to the shellcode, after which the shellcode gets executed and performs its task. Injecting the shellcode is often done by storing the shellcode in data sent over the network to the vulnerable process, by supplying it in a file that is read by the vulnerable process, or through the command line or environment in the case of local exploits.
It can be difficult or even impossible to write a shellcode into memory at the precise location to which program control will jump (i.e. at a location where the program expects to find some valid instruction) and/or to take control of the program counter. To overcome this problem, attackers have very successfully exploited a technique known as “heap spraying”. Briefly, heap spraying relies upon code within the downloaded data writing long strings of code, including multiple copies of the shellcode, into the heap memory, such that there is a high likelihood that a jump into the heap will result in the shellcode being executed. In the case of an attack exploiting a web browser, the heap spraying code is included as JavaScript code in the html web page. JavaScript code is executed by default by the web browser (or browser plugin) when it is present in a web page. [Of course, JavaScript is limited in what it can do to the system, so the actual malicious code is not implemented as JavaScript. Rather, JavaScript is merely used to write the shellcode to memory, and, if the program pointer can be caused to point to a copy of the shellcode, that is when the real damage is done.]
It can be fairly easy for an attacker to hide or obfuscate both the shellcode and the heap spraying code within a webpage, for example by using a “self-decoding” section of JavaScript. Using such an approach, the malicious JavaScript and shellcode is only “visible” once the JavaScript engine has been run, by which time the damage may be done. The self-decoding code may itself be varied so that it does not present a recognisable pattern. These measures together can defeat a simple scanning of the webpage for known malware signatures.
These and related issues are considered in the following Internet publications:    Heap Feng Shui in JavaScript, available as of Dec. 14, 2009 at http://phreedom.org/research/heap-feng-shui/heap-feng-shui.html Smashing the Stack for Fun and Profit by Aleph One (.o0 Phrack 49 o0o, volume severn issue forty-nine), available as of Dec. 14, 2009 at http://insecure.org/stf/smashstack.html    DaRk-CodeRs Group production, kid (English version, Sep. 3, 2008), available as of Dec. 14, 2009 at http://milw0rm.com/papers/205    X-Force Research & Development Newsletter (October 2006), available as of Dec. 14, 2009 at http://www.iss.net/documents/literature/X-ForceNews_Oct06.pdf